Convex optimization over intersection of simple sets: improved convergence rate guarantees via an exact penalty approach

We consider the problem of minimizing a convex function over the intersection of finitely many simple sets which are easy to project onto. This is an important problem arising in various domains such as machine learning. The main difficulty lies in finding the projection of a point in the intersection of many sets. Existing approaches yield an infeasible point with an iteration-complexity of $O(1/\varepsilon^2)$ for nonsmooth problems with no guarantees on the in-feasibility. By reformulating the problem through exact penalty functions, we derive first-order algorithms which not only guarantees that the distance to the intersection is small but also improve the complexity to $O(1/\varepsilon)$ and $O(1/\sqrt{\varepsilon})$ for smooth functions. For composite and smooth problems, this is achieved through a saddle-point reformulation where the proximal operators required by the primal-dual algorithms can be computed in closed form. We illustrate the benefits of our approach on a graph transduction problem and on graph matching

Fed+: A Unified Approach to Robust Personalized Federated Learning

We present a class of methods for robust, personalized federated learning, called Fed+, that unifies many federated learning algorithms. The principal advantage of this class of methods is to better accommodate the real-world characteristics found in federated training, such as the lack of IID data across parties, the need for robustness to outliers or stragglers, and the requirement to perform well on party-specific datasets. We achieve this through a problem formulation that allows the central server to employ robust ways of aggregating the local models while keeping the structure of local computation intact. Without making any statistical assumption on the degree of heterogeneity of local data across parties, we provide convergence guarantees for Fed+ for convex and non-convex loss functions and robust aggregation. The Fed+ theory is also equipped to handle heterogeneous computing environments including stragglers without additional assumptions; specifically, the convergence results cover the general setting where the number of local update steps across parties can vary. We demonstrate the benefits of Fed+ through extensive experiments across standard benchmark datasets as well as on a challenging real-world problem in financial portfolio management where the heterogeneity of party-level data can lead to training failure in standard federated learning approaches

Magnesium Contact Ions Stabilize the Tertiary Structure of Transfer RNA: Electrostatics Mapped by Two-Dimensional Infrared Spectra and Theoretical Simulations

Ions interacting with hydrated RNA play a central role in defining its secondary and tertiary structure. While spatial arrangements of ions, water molecules, and phosphate groups have been inferred from X-ray studies, the role of electrostatic and other noncovalent interactions in stabilizing compact folded RNA structures is not fully understood at the molecular level. Here, we demonstrate that contact ion pairs of magnesium (Mg2+) and phosphate groups embedded in local water shells stabilize the tertiary equilibrium structure of transfer RNA (tRNA). Employing dialyzed tRNAPhe from yeast and tRNA from Escherichia coli, we follow the population of Mg2+ sites close to phosphate groups of the ribose-phosphodiester backbone step by step, combining linear and nonlinear infrared spectroscopy of phosphate vibrations with molecular dynamics simulations and ab initio vibrational frequency calculations. The formation of up to six Mg2+/phosphate contact pairs per tRNA and local field-induced reorientations of water molecules balance the phosphate-phosphate repulsion in nonhelical parts of tRNA, thus stabilizing the folded structure electrostatically. Such geometries display limited sub-picosecond fluctuations in the arrangement of water molecules and ion residence times longer than 1 µs. At higher Mg2+ excess, the number of contact ion pairs per tRNA saturates around 6 and weakly interacting ions prevail. Our results suggest a predominance of contact ion pairs over long-range coupling of the ion atmosphere and the biomolecule in defining and stabilizing the tertiary structure of tRNA. © 2020 American Chemical Society

TOFA: Transfer-Once-for-All

Weight-sharing neural architecture search aims to optimize a configurable neural network model (supernet) for a variety of deployment scenarios across many devices with different resource constraints. Existing approaches use evolutionary search to extract a number of models from a supernet trained on a very large data set, and then fine-tune the extracted models on the typically small, real-world data set of interest. The computational cost of training thus grows linearly with the number of different model deployment scenarios. Hence, we propose Transfer-Once-For-All (TOFA) for supernet-style training on small data sets with constant computational training cost over any number of edge deployment scenarios. Given a task, TOFA obtains custom neural networks, both the topology and the weights, optimized for any number of edge deployment scenarios. To overcome the challenges arising from small data, TOFA utilizes a unified semi-supervised training loss to simultaneously train all subnets within the supernet, coupled with on-the-fly architecture selection at deployment time

Ultrafast vibrational response of activated C–D bonds in a chloroform–platinum(II) complex

[Image: see text] The vibrational response of the activated C–D bond in the chloroform complex [Pt(C(6)H(5))(2)(btz-N,N′)·CDCl(3), where btz = 2,2′-bi-5,6-dihydro-4H-1,3-thiazine] is studied by linear and nonlinear two-dimensional infrared (2D-IR) spectroscopy. The change of the C–D stretching vibration of metal-coordinated CDCl(3) relative to the free solvent molecule serves as a measure of the non-classical Pt···D–C interaction strength. The stretching absorption band of the activated C–D bond displays a red shift of 119 cm(–1) relative to uncoordinated CDCl(3), a strong broadening, and an 8-fold enhancement of spectrally integrated absorption. The infrared (IR) absorption and 2D-IR line shapes are governed by spectral diffusion on 200 fs and 2 ps time scales, induced by the fluctuating solvent CDCl(3). The enhanced vibrational absorption and coupling to solvent forces are assigned to the enhanced electric polarizability of the activated C–D bond. Density functional theory calculations show a significant increase of C–D bond polarizability of CDCl(3) upon coordination to the 16 valence electron Pt(II) complex

Convex optimization over intersection of simple sets: improved convergence rate guarantees via an exact penalty approach

International audienceWe consider the problem of minimizing a convex function over the intersection of finitely many simple sets which are easy to project onto. This is an important problem arising in various domains such as machine learning. The main difficulty lies in finding the projection of a point in the intersection of many sets. Existing approaches yield an infeasible point with an iteration-complexity of $O(1/\varepsilon^2)$ for nonsmooth problems with no guarantees on the in-feasibility. By reformulating the problem through exact penalty functions, we derive first-order algorithms which not only guarantees that the distance to the intersection is small but also improve the complexity to $O(1/\varepsilon)$ and $O(1/\sqrt{\varepsilon})$ for smooth functions. For composite and smooth problems, this is achieved through a saddle-point reformulation where the proximal operators required by the primal-dual algorithms can be computed in closed form. We illustrate the benefits of our approach on a graph transduction problem and on graph matching